OP-TEE in NVIDIA Jetson platforms

Table of Contents [Sticky]

This wiki shows how to implement OP-TEE for NVIDIA Jetson Platforms and will also guide you through implementing an example Trusted Application. This tutorial was tested with a Jetson Orin Nano module running JetPack 6; keep in mind that for other versions of JetPack, some steps may vary slightly. It is assumed that JetPack sources were already installed and contained in a directory used to initially flash the target board. Additionally, this tutorial will also show you how to implement OP-TEE in a Yocto build using the meta Tegra layer.

Implementation

As a first step, it is necessary to install the prerequisites defined by the official OP-TEE documentation, you can check them in this section. To install all prerequisites run the following commands:

apt update && apt upgrade

sudo apt install \
    adb \
    acpica-tools \
    autoconf \
    automake \
    bc \
    bison \
    build-essential \
    ccache \
    cpio \
    cscope \
    curl \
    device-tree-compiler \
    e2tools \
    expect \
    fastboot \
    flex \
    ftp-upload \
    gdisk \
    git \
    libattr1-dev \
    libcap-ng-dev \
    libfdt-dev \
    libftdi-dev \
    libglib2.0-dev \
    libgmp3-dev \
    libhidapi-dev \
    libmpc-dev \
    libncurses5-dev \
    libpixman-1-dev \
    libslirp-dev \
    libssl-dev \
    libtool \
    libusb-1.0-0-dev \
    make \
    mtools \
    netcat \
    ninja-build \
    python3-cryptography \
    python3-pip \
    python3-pyelftools \
    python3-serial \
    python3-tomli \
    python-is-python3 \
    rsync \
    swig \
    unzip \
    uuid-dev \
    wget \
    xdg-utils \
    xsltproc \
    xterm \
    xz-utils \
    zlib1g-dev

Then, you will need to download the Jetson-Linux toolchain from the Jetson release page corresponding to your L4T version, you can get it from the Jetson Linux Archive. In this guide, L4T 36.4.3 was used. After downloading, extract the toolchain in the folder where JetPack sources are located, use the following commands:

cd $Linux_for_Tegra_DIR
mkdir jetson-toolchain && cd jetson-toolchain
tar xf aarch64--glibc--stable-2022.08-1.tar.bz2 #The name of this file may change depending on the version

Info

Several of the steps shown in this guide will use a "<platform>" placeholder that needs to be replaced with the correct value for the platform you are using. Possible values are 234 for Jetson Orin platforms and 194 for Jetson Xavier platforms.

The following platforms are supported:

Supported platforms
Platform identifier	Description
t234	All Jetson Orin boards
t194	Jetson Xavier AGX board

Set environment variables

Some environment variables are required for the scripts that will be executed for the OP-TEE installation, you can set them with the following commands:

export CROSS_COMPILE_AARCH64_PATH=$Linux_for_Tegra_DIR/jetson-toolchain/aarch64--glibc--stable-2022.08-1

export CROSS_COMPILE_AARCH64=$CROSS_COMPILE_AARCH64_PATH/bin/aarch64-buildroot-linux-gnu-

export NV_TARGET_BOARD=generic

Then, you need to indicate the location of the UEFI StMM image, which is required to build OP-TEE, for all Jetson Orin platforms it is located in the path $Linux_for_Tegra_DIR/bootloader/standalonemm_optee_t234.bin. Set the environment variable UEFI_STMM_PATH to the path of the UEFI StMM image:

export UEFI_STMM_PATH=<StMM image path>

# Example:
export UEFI_STMM_PATH=$Linux_for_Tegra_DIR/bootloader/standalonemm_optee_t234.bin

Building the ATF source code

First, it is required to extract the JetPack sources:

cd $Linux_for_Tegra_DIR/source
source_sync.sh -t jetson_36.4.3 #Make sure to use your version of L4T

When the process finishes, OP-TEE sources will be located in the directory $Linux_for_Tegra_DIR/source/tegra/optee-src, and now you can build the ATF source code:

cd atf
./nvbuild.sh

After the process is completed, built files will be stored in the path $Linux_for_Tegra_DIR/source/tegra/optee-src/atf/arm-trusted-firmware/generic-t234/tegra/t234/release. This path applies for all Orin platforms.

Building the OP-TEE source code

Go to the OP-TEE sources, and then execute the script to build the source code:

cd $Linux_for_Tegra_DIR/source/tegra/optee-src/nv-optee/

./optee_src_build.sh -p t234

Building the OP-TEE dtb

In the same directory as the previous step, run the following command:

cd $Linux_for_Tegra_DIR/source/tegra/optee-src/nv-optee/
dtc -I dts -O dtb -o ./optee/tegra<platform>-optee.dtb ./optee/tegra<platform>-optee.dts

This will output something like this, which is a Warning from Linaro.

./optee/tegra194-optee.dts:33.17-38.4: Warning (unit_address_format): /efuse@03820000: unit name should not have leading 0s
./optee/tegra194-optee.dts:40.15-45.4: Warning (unit_address_format): /se0@03ac0000: unit name should not have leading 0s
./optee/tegra194-optee.dts:47.20-52.4: Warning (unit_address_format): /se0-rng1@03ae0000: unit name should not have leading 0s

The new dtb file will be located at $Linux_for_Tegra_DIR/source/tegra/optee-src/nv-optee/optee.

Generating the tos.img using ATF and OP-TEE sources

Go to the OP-TEE source code:

cd $Linux_for_Tegra_DIR/source/tegra/optee-src/

Move the gen_tos_part_img.py script to the current location. You can find the python script within $Linux_for_Tegra_DIR/nv_tegra/tos-scripts.

cp $Linux_for_Tegra_DIR/nv_tegra/tos-scripts/gen_tos_part_img.py .

Run:

./gen_tos_part_img.py --monitor ./atf/arm-trusted-firmware/generic-t234/tegra/t234/release/bl31.bin --os ./nv-optee/optee/build/t234/core/tee-raw.bin --dtb ./nv-optee/optee/tegra234-optee.dtb --tostype optee ./tos.img

At the end, the output should look like this:

Generating Trusted OS Partition Image File
Generate TOS Image File for boot-wrapper.

Two files should be created after this:

img.bin
tos.img

Flash the target platform

The recently generated tos.img should be renamed and relocated in order to start the flashing process. Go to the directory where tos.img is stored, and rename it as follows:

cd $Linux_for_Tegra_DIR/source/tegra/optee-src
mv tos.img tos_t<platform>.img

Where <platform> should be rename for one of the following options: 194 or 234.

Move the new img file to the bootloader directory. This will replace the old tos_t<platform>.img:

mv tos_t<platform>.img $Linux_for_Tegra_DIR/bootloader

Finally, proceed to flash the target board, you can do it with the following commands:

cd $Linux_for_Tegra_DIR
sudo ./flash.sh [options] <target_board> <rootdev>

#Example for a Jetson Orin Nano using an SD card:
sudo ./flash.sh jetson-orin-nano-devkit mmcblk0p1

Remember that before starting the flashing process the target board must be in recovery mode and connected to your host machine. In the case of the Jetson Orin Nano Devkit, you can set it to recovery mode by shorting pins 9 and 10 in the J14 header, the pins are labeled as FC_REC and GND respectively. After shorting the pins, you can plug the power adapter into the board and connect the board to your host machine using the USB C port.

After flashing and booting the board, copy all files in $Linux_for_Tegra_DIR/source/tegra/optee-src/nv-optee/optee/install/ to the board:

cd $Linux_for_Tegra_DIR/source/tegra/optee-src/nv-optee/optee/install/
scp -r t234 nvidia@address:/~

After this, move the files to their corresponding locations. The t234 folder contains three folders: bin, lib and usr. Copy the content of each folder to the corresponding root filesystem locations.

Adding Trusted Applications

As an example of how to add and execute a Trusted Application, we will use the Hello World example from the official OP-TEE documentation. This is a simple example to illustrate how a CA in the normal world can communicate with its corresponding TA in the secure world. The CA defines defines a single integer parameter and then calls the TA to either increase or decrease the parameter by one. The increase or decrease operation is performed by the TA and the CA is simply in charge of calling the TA and printing the result. By default the increase operation is performed

The code for the CA is the following:

/*
 * Copyright (c) 2016, Linaro Limited
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 * 1. Redistributions of source code must retain the above copyright notice,
 * this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above copyright notice,
 * this list of conditions and the following disclaimer in the documentation
 * and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 */

#include <err.h>
#include <stdio.h>
#include <string.h>

/* OP-TEE TEE client API (built by optee_client) */
#include <tee_client_api.h>

/* For the UUID (found in the TA's h-file(s)) */
#include <hello_world_ta.h>

int main(void)
{
	TEEC_Result res;
	TEEC_Context ctx;
	TEEC_Session sess;
	TEEC_Operation op;
	TEEC_UUID uuid = TA_HELLO_WORLD_UUID;
	uint32_t err_origin;

	/* Initialize a context connecting us to the TEE */
	res = TEEC_InitializeContext(NULL, &ctx);
	if (res != TEEC_SUCCESS)
		errx(1, "TEEC_InitializeContext failed with code 0x%x", res);

	/*
	 * Open a session to the "hello world" TA, the TA will print "hello
	 * world!" in the log when the session is created.
	 */
	res = TEEC_OpenSession(&ctx, &sess, &uuid,
			       TEEC_LOGIN_PUBLIC, NULL, NULL, &err_origin);
	if (res != TEEC_SUCCESS)
		errx(1, "TEEC_Opensession failed with code 0x%x origin 0x%x",
			res, err_origin);

	/*
	 * Execute a function in the TA by invoking it, in this case
	 * we're incrementing a number.
	 *
	 * The value of command ID part and how the parameters are
	 * interpreted is part of the interface provided by the TA.
	 */

	/* Clear the TEEC_Operation struct */
	memset(&op, 0, sizeof(op));

	/*
	 * Prepare the argument. Pass a value in the first parameter,
	 * the remaining three parameters are unused.
	 */
	op.paramTypes = TEEC_PARAM_TYPES(TEEC_VALUE_INOUT, TEEC_NONE,
					 TEEC_NONE, TEEC_NONE);
	op.params[0].value.a = 42;

	/*
	 * TA_HELLO_WORLD_CMD_INC_VALUE is the actual function in the TA to be
	 * called.
	 */
	printf("Invoking TA to increment %d\n", op.params[0].value.a);
	res = TEEC_InvokeCommand(&sess, TA_HELLO_WORLD_CMD_INC_VALUE, &op,
				 &err_origin);
	if (res != TEEC_SUCCESS)
		errx(1, "TEEC_InvokeCommand failed with code 0x%x origin 0x%x",
			res, err_origin);
	printf("TA incremented value to %d\n", op.params[0].value.a);

	/*
	 * We're done with the TA, close the session and
	 * destroy the context.
	 *
	 * The TA will print "Goodbye!" in the log when the
	 * session is closed.
	 */

	TEEC_CloseSession(&sess);

	TEEC_FinalizeContext(&ctx);

	return 0;
}

The CA first connects to the TEE of the device. If it was able to correctly connect to the TEE, it will create a session for the corresponding TA, which is identified by its UUID. Then it will call the TA to perform the operation and finally it finished the TA session and closes the connection to the TEE.

For the TA, it is defined as follows:

/*
 * Copyright (c) 2016, Linaro Limited
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 * 1. Redistributions of source code must retain the above copyright notice,
 * this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above copyright notice,
 * this list of conditions and the following disclaimer in the documentation
 * and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 */

#include <tee_internal_api.h>
#include <tee_internal_api_extensions.h>

#include <hello_world_ta.h>

/*
 * Called when the instance of the TA is created. This is the first call in
 * the TA.
 */
TEE_Result TA_CreateEntryPoint(void)
{
	DMSG("has been called");

	return TEE_SUCCESS;
}

/*
 * Called when the instance of the TA is destroyed if the TA has not
 * crashed or panicked. This is the last call in the TA.
 */
void TA_DestroyEntryPoint(void)
{
	DMSG("has been called");
}

/*
 * Called when a new session is opened to the TA. *sess_ctx can be updated
 * with a value to be able to identify this session in subsequent calls to the
 * TA. In this function you will normally do the global initialization for the
 * TA.
 */
TEE_Result TA_OpenSessionEntryPoint(uint32_t param_types,
		TEE_Param __maybe_unused params[4],
		void __maybe_unused **sess_ctx)
{
	uint32_t exp_param_types = TEE_PARAM_TYPES(TEE_PARAM_TYPE_NONE,
						   TEE_PARAM_TYPE_NONE,
						   TEE_PARAM_TYPE_NONE,
						   TEE_PARAM_TYPE_NONE);

	DMSG("has been called");

	if (param_types != exp_param_types)
		return TEE_ERROR_BAD_PARAMETERS;

	/* Unused parameters */
	(void)&params;
	(void)&sess_ctx;

	/*
	 * The DMSG() macro is non-standard, TEE Internal API doesn't
	 * specify any means to logging from a TA.
	 */
	IMSG("Hello World!\n");

	/* If return value != TEE_SUCCESS the session will not be created. */
	return TEE_SUCCESS;
}

/*
 * Called when a session is closed, sess_ctx hold the value that was
 * assigned by TA_OpenSessionEntryPoint().
 */
void TA_CloseSessionEntryPoint(void __maybe_unused *sess_ctx)
{
	(void)&sess_ctx; /* Unused parameter */
	IMSG("Goodbye!\n");
}

static TEE_Result inc_value(uint32_t param_types,
	TEE_Param params[4])
{
	uint32_t exp_param_types = TEE_PARAM_TYPES(TEE_PARAM_TYPE_VALUE_INOUT,
						   TEE_PARAM_TYPE_NONE,
						   TEE_PARAM_TYPE_NONE,
						   TEE_PARAM_TYPE_NONE);

	DMSG("has been called");

	if (param_types != exp_param_types)
		return TEE_ERROR_BAD_PARAMETERS;

	IMSG("Got value: %u from NW", params[0].value.a);
	params[0].value.a++;
	IMSG("Increase value to: %u", params[0].value.a);

	return TEE_SUCCESS;
}

static TEE_Result dec_value(uint32_t param_types,
	TEE_Param params[4])
{
	uint32_t exp_param_types = TEE_PARAM_TYPES(TEE_PARAM_TYPE_VALUE_INOUT,
						   TEE_PARAM_TYPE_NONE,
						   TEE_PARAM_TYPE_NONE,
						   TEE_PARAM_TYPE_NONE);

	DMSG("has been called");

	if (param_types != exp_param_types)
		return TEE_ERROR_BAD_PARAMETERS;

	IMSG("Got value: %u from NW", params[0].value.a);
	params[0].value.a--;
	IMSG("Decrease value to: %u", params[0].value.a);

	return TEE_SUCCESS;
}
/*
 * Called when a TA is invoked. sess_ctx hold that value that was
 * assigned by TA_OpenSessionEntryPoint(). The rest of the paramters
 * comes from normal world.
 */
TEE_Result TA_InvokeCommandEntryPoint(void __maybe_unused *sess_ctx,
			uint32_t cmd_id,
			uint32_t param_types, TEE_Param params[4])
{
	(void)&sess_ctx; /* Unused parameter */

	switch (cmd_id) {
	case TA_HELLO_WORLD_CMD_INC_VALUE:
		return inc_value(param_types, params);
	case TA_HELLO_WORLD_CMD_DEC_VALUE:
		return dec_value(param_types, params);
	default:
		return TEE_ERROR_BAD_PARAMETERS;
	}
}

It includes a function to interpret which operation was requested and a function for each operation, only one of these functions will be called depending on the operation requested by the CA. Apart from this, standard functions are included to handle the TA sessions and entry points. These standard functions must be defined in every TA as are required by the API to handle the communication between the normal world and the secure world.

To add the trusted application, clone the OP-TEE examples repository in your host machine:

git clone https://github.com/linaro-swg/optee_examples.git

Then, copy the hello_world example to the OP-TEE sources:

cd $HOME/optee_examples 
cp -r hello_world $Linux_for_Tegra_DIR/source/tegra/optee-src/nv-optee/optee/samples

Modify the TA root Makefile. Go to the hello_world directory within the sources:

cd $Linux_for_Tegra_DIR/source/tegra/optee-src/nv-optee/optee/samples/hello_world
sudo nano Makefile

Add the following changes:

-# If _HOST or _TA specific compilers are not specified, then use CROSS_COMPILE
-HOST_CROSS_COMPILE ?= $(CROSS_COMPILE)
-TA_CROSS_COMPILE ?= $(CROSS_COMPILE)
-
-.PHONY: all
-all:
-	$(MAKE) -C host CROSS_COMPILE="$(HOST_CROSS_COMPILE)" --no-builtin-variables
-	$(MAKE) -C ta CROSS_COMPILE="$(TA_CROSS_COMPILE)" LDFLAGS=""

-.PHONY: clean
-clean:
-	$(MAKE) -C host clean
-	$(MAKE) -C ta clean

+TARGET_DIR := $(notdir $(shell pwd))
+
+.PHONY: all
+all:
+	$(MAKE) -C ta \
+		CROSS_COMPILE=$(CROSS_COMPILE) \
+		TA_DEV_KIT_DIR=$(TA_DEV_KIT_DIR) \
+		O=$(O)/ta/$(TARGET_DIR)
+	$(MAKE) -C host \
+		CROSS_COMPILE=$(CROSS_COMPILE) \
+		OPTEE_CLIENT_EXPORT=$(OPTEE_CLIENT_EXPORT) \
+		O=$(O)/ca/$(TARGET_DIR) \
+		--no-builtin-variables

+.PHONY: clean
+clean:
+	$(MAKE) -C ta \
+		TA_DEV_KIT_DIR=$(TA_DEV_KIT_DIR) \
+		O=$(O)/ta/$(TARGET_DIR) \
+		clean
+	$(MAKE) -C host \
+		OPTEE_CLIENT_EXPORT=$(OPTEE_CLIENT_EXPORT) \
+		O=$(O)/ca/$(TARGET_DIR) \
+		clean
+	rm -rf $(O)/ca/$(TARGET_DIR)

Modify the Makefile within the host directory.

cd /host
sudo nano Makefile

Apply the following changes:

-CC      ?= $(CROSS_COMPILE)gcc
-LD      ?= $(CROSS_COMPILE)ld
-AR      ?= $(CROSS_COMPILE)ar
-NM      ?= $(CROSS_COMPILE)nm
-OBJCOPY ?= $(CROSS_COMPILE)objcopy
-OBJDUMP ?= $(CROSS_COMPILE)objdump
-READELF ?= $(CROSS_COMPILE)readelf
-
-OBJS = main.o
-
-CFLAGS += -Wall -I../ta/include -I$(TEEC_EXPORT)/include -I./include
-Add/link other required libraries here
-LDADD += -lteec -L$(TEEC_EXPORT)/lib
-
-BINARY = optee_example_hello_world
-
-.PHONY: all
-all: $(BINARY)
-
-$(BINARY): $(OBJS)
-	$(CC) $(LDFLAGS) -o $@ $< $(LDADD)
-
-.PHONY: clean
-clean:
-	rm -f $(OBJS) $(BINARY)

-%.o: %.c
-	$(CC) $(CFLAGS) -c $< -o $@
+
+CC ?= $(CROSS_COMPILE)gcc
+
+CFLAGS += -Wall -I../ta/include -I./include
+CFLAGS += -I$(OPTEE_CLIENT_EXPORT)/include
+CFLAGS += -fstack-protector-strong
+LDADD += -lteec -L$(OPTEE_CLIENT_EXPORT)/lib
+
+SRCS = main.c
+OBJS = $(patsubst %.c,$(O)/%.o,$(SRCS))
+BINARY = optee_example_hello_world
+
+.PHONY: all install
+all: $(BINARY) install
+
+$(BINARY): $(OBJS)
+	$(CC) -o $(O)/$@ $< $(LDADD)
+
+$(O)/%.o: %.c
+	mkdir -p $(O)
+	$(CC) $(CFLAGS) -c $< -o $@
+
+install: $(BINARY)
+	mkdir -p $(OPTEE_CLIENT_EXPORT)/sbin
+	cp $(O)/$(BINARY) $(OPTEE_CLIENT_EXPORT)/sbin
+
+.PHONY: clean
+clean:
+	rm -f $(OBJS) $(O)/$(BINARY) $(OPTEE_CLIENT_EXPORT)/sbin/$(BINARY)

Go to the /ta directory and make sure that the sub.mk includes the source and header file for the TA.

cd ../ta/
cat sub.mk

It should look like this:

global-incdirs-y += include
srcs-y += hello_world_ta.c

The file hello_world_ta.c is the source code and hello_world_ta.h lives within the include directory. Modify the Makefile inside the /ta directory. Apply the next changes

-CFG_TEE_TA_LOG_LEVEL ?= 4
-CFG_TA_OPTEE_CORE_API_COMPAT_1_1=y

- The UUID for the Trusted Application
-BINARY=8aaaf200-2450-11e4-abe2-0002a5d5c51b

-include $(TA_DEV_KIT_DIR)/mk/ta_dev_kit.mk

-ifeq ($(wildcard $(TA_DEV_KIT_DIR)/mk/ta_dev_kit.mk), )
-clean:
-	@echo 'Note: $$(TA_DEV_KIT_DIR)/mk/ta_dev_kit.mk not found, cannot clean TA'
-	@echo 'Note: TA_DEV_KIT_DIR=$(TA_DEV_KIT_DIR)'
-endif

- Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
- SPDX-License-Identifier: BSD-2-Clause
+
+# The UUID for the hello world TA
+BINARY=8aaaf200-2450-11e4-abe2-0002a5d5c51b
+
+include $(TA_DEV_KIT_DIR)/mk/ta_dev_kit.mk

The previous changes will allow the code to build correctly by redirecting include statements to the correct paths in the directory structure of the OP-TEE sources being used, and redirect the build output to the install folder. After that, build the sources again:

cd $Linux_for_Tegra_DIR/source/tegra/optee-src/nv-optee/

./optee_src_build.sh -p t234

The build process will output an optee_example_hello_world binary (from the host Makefile), and a uuid.{ta,elf,dmp,map} (from the ta Makefile). They can be found in this location:

/$Linux_for_Tegra_DIR/source/tegra/optee-src/nv-optee/optee/build/t234

Build and flash the OP-TEE image and dtb again, as shown before.

Adding a single application

With the previous approach, all Trusted Applications in the sample directory will be built and installed at once, which makes it convenient for an initial deployment. However, if you are adding a single new Trusted Application or modifying a previously deployed one, it is not convenient to build all the sources again and need to flash the board.

For a case like this, you can build a single application with the following command:

make -C <source directory> \
     CROSS_COMPILE="<jetson-toolchain>/bin/aarch64-buildroot-linux-gnu-" \
     TA_DEV_KIT_DIR="<jetson-optee-srcs>/optee/build/t234/export-ta_arm64/" \
     OPTEE_CLIENT_EXPORT="<jetson-optee-srcs>/optee/install/t234/usr" \
     TEEC_EXPORT="<jetson-optee-srcs>/optee/install/t234/usr" \
     -j"$(nproc)"

The previous command will create the executables for the application located in the source folder. When the process finishes, you need to place the corresponding files in your board. The CA must be placed in the path /usr/sbin, while the TA must be in /lib/optee_armtz. After this you will be able to execute your new Trusted Application.

Run the new Trusted Application

After building the application, copy the generated files to the Jetson platform. Go to the install directory and look for a t234 directory. Send it to the target platform.

cd $Linux_for_Tegra_DIR/source/tegra/optee-src/nv-optee/optee/install/
scp -r t234 nvidia@address:/~

On the Jetson platform. Place the files and directories inside it at their respective locations on the Jetson.

Then, you will be able to execute the trusted application.

sudo optee_example_hello_world

# Output
Invoking TA to increment 42
TA incremented value to 43

In the previous snippet the application was executed successfully as the corresponding TA exists and it is correctly called by the optee_example_hello_world CA. You can verify by forcing the execution to fail by deleting the TA, for this particular example it is 8aaaf200-2450-11e4-abe2-0002a5d5c51b.ta and it is located in the path /lib/optee_armtz with other installed TAs. You can delete it with the following command:

cd lib/optee_armtz

sudo rm 8aaaf200-2450-11e4-abe2-0002a5d5c51b.ta

After that, if you try to execute the CA you will get the following output, as it will not be able to communicate with the created TEE:

sudo optee_example_hello_world

# Output
optee_example_hello_world: TEEC_Opensession failed with code 0xffff0008 origin 0x3

As mentioned previously, by default the example will increase the provided argument, but the example TA also included a function to decrease the supplied argument. Initially, the code snippet of the CA where the TA function is called looks like this:

printf("Invoking TA to increment %d\n", op.params[0].value.a);
res = TEEC_InvokeCommand(&sess, TA_HELLO_WORLD_CMD_INC_VALUE, &op, &err_origin);

if (res != TEEC_SUCCESS)
	errx(1, "TEEC_InvokeCommand failed with code 0x%x origin 0x%x", res, err_origin);

printf("TA incremented value to %d\n", op.params[0].value.a);

Where TA_HELLO_WORLD_CMD_INC_VALUE is the function in the TA that will be invoked by the TEEC_InvokeCommand function. If you want the decode to decrease the provided value instead of increasing it, you can modify the previous code snippet to the following:

printf("Invoking TA to decrease %d\n", op.params[0].value.a);
res = TEEC_InvokeCommand(&sess, TA_HELLO_WORLD_CMD_DEC_VALUE, &op, &err_origin);

if (res != TEEC_SUCCESS)
	errx(1, "TEEC_InvokeCommand failed with code 0x%x origin 0x%x", res, err_origin);

printf("TA decreased value to %d\n", op.params[0].value.a);

You can also modify the Makefile in the host directory so that the created binary will have a different name and you can differentiate between the two cases:

BINARY = optee_example_hello_world_decrease

Then, after building the code and placing the new binary in the /usr/sbin path of your board you will have both cases ready to execute:

sudo optee_example_hello_world

# Output
Invoking TA to increment 42
TA incremented value to 43

sudo optee_example_hello_world_decrease

# Output
Invoking TA to decrease 42
TA decreased value to 41

Additionally, automated tests are included with the OP-TEE integration. You can use these tests to verify that the integration is correct with the xtest command. For example:

xtest

# Example final output
35360 subtests of which 0 failed
136 test cases of which 0 failed
0 test cases were skipped
TEE test application done!

Yocto Support

If you are using a custom Yocto build in your Jetson board, implementing OP-TEE with the meta tegra layer is possible. This implementation was tested using a Jetson Orin Nano Devkit. As a first step you can follow RidgeRun's Yocto Guide for NVIDIA® Jetson™ with JetPack 6 Integration. To obtain the sources and the base configuration files. After that, simply add the following features to your build by adding the following in your local.conf file:

IMAGE_INSTALL:append ="\
    optee-os \
    optee-client \
    optee-nvsamples \
    optee-test \
    arm-trusted-firmware \
"

After this, you can procede to build your image, for example:

bitbake -k core-image-sato

After completing the building process, you will have a .tegraflash.tar.gz file in $YOCTO_DIR/build/tmp/deploy/images/$MACHINE. The .tegraflash.tar.gz file will be used to flash your board for the first time. To flash the board, it is recommended to use a temporary directory to decompress the .tegraflash.tar.gz file, you can use the following commands to do so:

tmpdir=`mktemp`
rm -rf $tmpdir
mkdir -p $tmpdir
pushd $tmpdir
cp $YOCTO_DIR/build/tmp/deploy/images/$MACHINE/$IMAGE .
tar -xvf $IMAGE

With the Jetson Orin Nano and a core-image-sato image the copy and tar commands look like the following:

cp $YOCTO_DIR/build/tmp/deploy/images/jetson-orin-nano-devkit/core-image-sato-jetson-orin-nano-devkit.tegraflash.tar.gz .
tar -xvf core-image-sato-jetson-orin-nano-devkit.tegraflash.tar.gz

Before starting the flashing process, as described in the previous sections. After that, you can start the flashing process by executing the following command:

sudo ./doflash.sh

When the flashing process is completed, your board will be enabled by OP-TEE. You can verify this with the xtest command just like in the previous case:

xtest

# Example final output
35360 subtests of which 0 failed
136 test cases of which 0 failed
0 test cases were skipped
TEE test application done!

❯